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Welcome to Education Development Trust 


At Education Development Trust, we have been improving education around the 
world for 50 years. We design and implement improvement programmes for school 
systems, and provide consultancy services deploying specialists internationally. 
Our work is informed by our continually refreshed body of research which focuses 
on the bright spots in education, from education authorities as diverse as those in 
Vietnam, Kenya, England, New York and Dubai. 


Bringing about real change that alters the aspects of a national system that, 

for many reasons, aren't working so well at the time, requires knowledge and 
ability to design and implement changes to any of the levers that can impede 
great educational outcomes. So the ability to affect policy, practices, pedagogy, 
behaviour, funding, attitudes and more is a prerequisite for a company that can 
truly claim to transform lives through improving education. 


As highly informed agents of change operating in low- to high-income countries 
with their varying internal contexts, we not only design but also show and enable, 
so when working with us, everyone involved, from policymakers to school 
leaders and teachers, is able to apply their new knowledge to drive sustainable 
system reform. 


Our expert knowledge, programme design and implementation expertise is also 
deployed in delivering Ofsted-rated outstanding careers services in England, 
and in owning and managing a family of independent schools. 


We are a not-for-profit and we are driven by our values of integrity, accountability, 
excellence and collaboration. 
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About the authors 


Martin Kalizewski is an education consultancy professional with over 17 years’ 
experience of leading and managing the use of school performance data in 
government local authorities, England as a whole and several international 
contexts. He has been a significant contributor to the English national data 
system, known as RAISEonline, which is used by schools and Ofsted inspectors. 
This web-based, pupil-level data driven analysis system was the first of its kind 
in the world. Known in the UK as RAISEonline, it provides detailed reporting to 
enable approximately 20,000 schools to evaluate their performance in support 
of their self-evaluation and to provide inspectors with the data necessary to 
inform school inspection. 


Martin trained as a teacher for both primary and secondary sectors, and he has 
taught in a range of secondary schools in England. Initially a mathematics teacher, 
he quickly moved into senior school leadership roles before becoming a School 
Adviser for East Sussex local authority, where he supported school improvement 
across all phases of education and all types of schools. He was responsible 

for monitoring schools’ performance and setting targets for improvement. In 
2006, Martin was seconded to the UK government's Department for Education 

to take on this role in a national context through his work developing the 
RAISEonline system. 


Martin has also worked closely with the Fischer Family Trust (now known as FFT) to 
help them develop the UK's national system for school self-evaluation reporting. 


As a Principal Education Consultant at Education Development Trust, Martin 
travelled to Brunei to advise and train the Ministry of Education on data systems. 
He also travelled to Rwanda to provide analysis of the current data architecture 
of the Rwandan Ministry of Education and Rwanda Education Board; developing 
solutions for an analytics platform; developing solutions for mini-assessments 
in Rwandan primary schools; and developing programme monitoring solutions 
aligned to the data architecture vision. Martin also completed an initial scoping 
exercise for the sampling methodology for the monitoring and evaluation 
component. 


Astrid Fieldsend has spent most of her professional life teaching in challenging 
secondary schools in London in a variety of roles up to Senior Leadership level. 
While teaching, she worked on education projects for a range of organisations 
including Google and the Varkey Foundation, as well as undertaking an MA in 
Education Leadership at UCL Institute of Education, developing experience across 
the sector. Astrid’s interest in England's use of data stems from her time spent in 
school, particularly while in roles focused on raising achievement. 


Astrid joined Education Development Trust in February 2017. Since joining, she 
has combined business development and project management within the 
Research and Consultancy team, as well as carrying out technical assignments 
in the UK and overseas. 
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Foreword 


Every year the World Bank produces the World Development Report! (WDR) 
analysing some of the key challenges facing humanity. The WDR for 2018 focuses 
exclusively on the challenges that the world faces in the field of education, and 
highlighted the urgent need for good data about student learning outcomes within 
any improving government school system. The authors of the WDR 2018 remind 
us of the challenge we face as a global community in our attempts to ensure that 
all students have access to a high-quality school education as measured by good 
learning outcomes. They recommend three inter-connected and complementary 
strategies needed to drive better outcomes: 


Assess learning—to make it a serious goal. Measure and track learning better; 
use the results to guide action. 


Act on evidence—to make schools work for all learners. Use evidence to guide 
innovation and practice. 


Align actors—to make the whole system work for learning. Tackle the technical 
and political barriers to learning at scale. 


The WDR 2018 talked about the need for measurement that shines a light on 
learning. Educational data is the fuel for the engine of school improvement. 

By itself data changes nothing but, properly used, data can stimulate beneficial 
change at every level: from the dialogue between a teacher and an individual 
student to the decisions made national policymakers about priorities for 
educational reform. 


The first step to improving systemwide learning is to put in place good metrics 
for monitoring whether programs and policies are delivering learning. Credible, 
reliable information can shape the incentives facing politicians. Most notably, 
information on student learning and school performance—if presented in a way 
that makes it salient and acceptable—fosters healthier political engagement 
and better service delivery. Information also helps policy makers manage a 
complex system. Measuring learning can improve equity by revealing hidden 
exclusions.* 


Experience from England supports this analysis and provides a case study for the 
first of the three recommended strategies: the systematic tracking of learning 
outcomes and the use of the resulting data as a guide to action at every level of 
the system. 


This report tells the story of how the national Education Management Information 
System has evolved in England. They key components of the system are a regime 
of reliable tests taken by all students in all government schools at ages 11 and 

16 and a National Pupil Database which links the test results to the background 

of each student. This link between background and academic outcomes is made 
possible through the use of the Unique Pupil Number (UPN). The UPN supports 

a comprehensive electronic record of the background every student in terms of 
gender, ethnicity, first language, poverty levels and special educational needs. 


‘World Bank, 2018 *Ibid., p.16 
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It is therefore possible to track the performance of students from different 
backgrounds and to benchmark the outcomes for every school, district and region 
in the country. 


In the following pages the evolution of this system is described. One key feature 

is a high degree of public accountability based on data transparency. Official data 
on the performance of schools is shared with parents and the wider community. 
Test results for each school for 11 year-olds and 16 year-olds are published and 
since 1992 performance tables showing the results for all schools are made public. 
The publication of test results is closely associated with the publication of school 
inspection data and inspection reports. 


The data system in England is heavily dependent on computer-based technology. 
Public data is hosted on websites. Confidential data is also available for school 
staff, school inspectors and others through a password-protected passport. 


The development of the data system in England has not been without problems 
and some controversy. Publishing performance data can have unintended and 
sometimes negative consequences. Many teachers consider that there is too much 
data-based accountability in England and not enough support for their professional 
development. 


The report describes some of the debates that have taken place in recent 

years. Should we use simple binary pass or fail measures or are there more 
comprehensive measures of learning outcomes? How should we address the fact 
that schools serving different communities will often have a different learning 
baseline? Should there be measures of progress and ‘value-added’ as well as ‘raw 
performance scores? 


One of the most important, and positive aspects, of the story told in this report 
is the way that the data system has been used for internal purposes of school 
self-evaluation and school development planning. A good data system can 
contribute to school improvement both through assisting external accountability 
and internally-driven changes led by school leaders. Our own research into the 
improvement of schools in London and the rapid turnaround of some schools in 
England has indicated how important data-based self-evaluation can be. 


The English approach is not presented here as a blueprint for other countries. 
Every education system needs its own unique rote map to between student 
outcomes. While we are not advocating that people in other countries simply copy 
English practice, we are confident that policymakers worldwide will find aspects of 
the data story in England interesting and instructive. 


Tony McAleavy 
Director of Research and Consultancy 
Education Development Trust 
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CHAPTER 1: INTRODUCTION 


This report focuses on the development of 
England's national Education Management 
Information System. 


Data has played an important role in England's recent school improvement 
journey. The evolution of the approach has not been perfect but the National Pupil 
Database has become a vital tool for health checking the education system, driving 
accountability, directing education policymaking and tracking the educational 
attainment of key vulnerable groups. 


The national data 
set in England 
now includes 
information about 


Within the United Kingdom, education is a ‘devolved’ responsibility managed at the tens of millions 
level of each of the four national jurisdictions: England, Northern lreland, Scotland of current and 
and Wales. This report focuses on experiences in England, which is by far the former school 
largest of the four political units. There are more than 20,000 government schools pupils; for 

in England and more than a half million pupils in each year group. those who have 


graduated from 
the school system 
the data can be 


The school system in England is highly accountable. This has not always been 
the case but during the years 1988-1992 a relatively centralised system of 


accountability was put in place based on: 
used to track 


° new tests for pupils in all primary schools progress during 


* the publication of results of primary and secondary school test results their entire school 
career 


¢ a new tough school inspection agency, called Ofsted, which published its regular 
reports in to school quality. 


The national accountability system evolved further in 1999-2002. A system of 
unique ‘identifiers’ were established for all pupils in government schools. This 
Unique Pupil Number (UPN), which was established in 1999, is a focal point for 
key data associated with every single student who goes to a government school. 
The value of the UPN system was enhanced in 2002 when a comprehensive survey 
of all students was established known as the Pupil Level Annual Schools Census 
(PLASC). Since then, the government in England has collected individual pupil 
data from all government schools for all pupils. This has been used to create the 
National Pupil Database (NPD): a rich and extensive longitudinal data set which is 
analysed by researchers, policy makers, national and local government officers, 
school inspectors and the schools themselves. 


The national data set in England now includes information about tens of millions of 
current and former school pupils; for those who have graduated from the school 
system the data can be used to track progress during their entire school career. 
This data set is comprehensive, including all government schools. The information 
has, over time, become more detailed in content and enabled more sophisticated 
analysis. As a result, a considerable amount of information is now held about pupils 
‘tagged’ to each individual's UPN. The UPNs act as identifiers allowing pupils to be 
tracked regardless, for example, of whether their families move to different parts of 
England during their school career. The UPN enables detailed data about individual 
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pupils’ progress to be collated and analysed. The system also facilitates the 
generation of data about the academic progress of specific cohorts of students (for 
example, girls versus boys, those living in poverty or those with special educational 
needs) to be extrapolated for analysis and tracking. 


The system is now data rich. In addition to the national data set, schools also 
undertake considerable additional data collection about the performance of their 
own pupils. This is done for the purpose of tracking performance against targets 
and learning criteria. Additional school level information is not regulated by the 
government or any other national bodies, and is used by schools for their own 
internal diagnostic purposes. It is an expectation, however, that schools track and 
monitor pupils in this way and are able to use the rich data collected to assist with 
the school self-evaluation and improvement planning process. 


Technology plays a key role in underpinning the national and more localised 
school collection, storing and analysis of data. The collection and processing of 
this data has been enabled by the widespread proliferation of lower cost computers 
and software available to the sector since the launch of personal computer and 
software, based upon graphical user interfaces both of which were used in schools 
from the mid-1980s and became more widespread in the 1990s. 


This report seeks to provide insights and lessons from the 
collection and use of data in England in recent years 


The purpose of this report is to highlight some of the lessons learned and 
successes of the England national pupil data story. This report is based on the 
expertise and experience of its authors and draws on some relevant literature 

to support key points and provide illustrations. We do not intend to provide a 
detailed history or completely comprehensive account of the story of English 
education data. It is not our intention to suggest that England's system should 

be seen as a perfect example and used simplistically as a blueprint for others. 
Instead, this report is intended to offer insights for policymakers in other countries 
where a national pupil level data system is being developed or refined. By sharing 
England's story we hope also that others may be able to avoid some of the 
mistakes and unexpected consequences that were made in the last three decades 
of development in England. 


Our report provides analysis of seven key lessons 


1. The school accountability system in England benefits from a data infrastructure 
of national tests and information about pupil characteristics, supported by 
technology. 


2. English experience demonstrates the risks of an over-simplistic approach to 
school performance data. 


3. In England, there has been an important and lively debate about ‘value-added’ 
measures which take into account the starting points of pupils. 
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4. There is a need for a national accountability system that evolves over time. 


5. Accountability measures used in England have played a significant role in the 
development of school self-evaluation. 


6. Schools can benefit from comparing their own students’ performance with that 
of the national database through item level analysis. 


7. There should be an alignment between data-based support for internal school 
improvement and external school improvement through inspection. 
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Lesson 1: The school accountability 
system in England benefits froma data 
infrastructure of national tests and 
information about pupil characteristics, 
supported by technology. 


In recent years, the prime purpose for school performance data in England 

has been to enable schools to be held to account and, through data-driven 
accountability, to generate an increase in educational standards. Whilst the data 
is also available to researchers and academics, this access is additional to the 
prime need to provide public accountability for the performance of government 
schools funded through public taxation. The data system has enabled successive 
governments to drive an agenda of national reform and to attempt to improve 
teaching and learning. 


The data system 
has enabled 
successive 
governments to 


drive an agenda 
of national reform 
and to attempt to 
The current national data infrastructure in England has, as an essential component, improve teaching 
high stakes tests which can be analysed at the level of the individual pupil, the and learning 
school, the district, the region and the whole country. 


The government defines performance indicators for school quality and 

publishes these in an annual ‘Statement of Intent’. Educational data is politicised. 
Performance indicators are decided by ministers and, of course, often reflect their 
political priorities. Ministers may, for example, be of the view that some subjects 
are more important than others and choose to emphasise these subjects ina 
measure that they approve. Past ministers have remarked that these accountability 
measures are one of the ‘big national levers of change’ available to them.’ 


In England current examples of government indicators are based on 
expectations that: 


Pupils should attain an ‘expected standard’ at the end of primary schools (Key 
Stage 2, age 11) in reading, writing and mathematics. 


Pupils should attain a ‘good’ outcome at the end of secondary school (Key Stage 
4, age 16) in the five subject areas termed collectively ‘the English Baccalaureate’ 
(English, maths, the sciences, history or geography and a language).* 


The purpose of setting such expectations and using these as public indicators of 
performance is to incentivise school leaders and teachers to bring about good 
student outcomes as defined by these measures. Much school performance data 
is placed in the public domain. The system is highly transparent. The systematic 
publication of ‘performance tables’ for public examination and test results, 

which began in 1992, is now an established feature of the educational system in 
England. The performance tables are sometimes referred to as ‘league tables’ The 
publication of results in 1992 coincided with the establishment of the national 


5Estelle Morris, 2014 *DfE, 2016a 
eee eee ee ee eee ee eT TT TT Te ee Te eT Te TT TT TT TT TTT TT TT TT TT TT TT TTT TT TT TT TT TT TT TT eee TT a 
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school inspectorate, Ofsted, which publishes its inspection findings. Published test 
results and published inspection reports have, ever since 1992, been part of the 
accountability infrastructure. 


In England, all pupils attending government schools take part in tests and other 
national assessment activities throughout their school careers. The most important 
assessment measures are tests undertaken by almost all 11 year olds at the end of 
Year 6, and examinations (known as the GCSE: the General Certificate of Secondary 
Education) undertaken by most 16 year olds at the end of Year 11. The school level 
results for these tests and examinations are published. 


With regard to data from these tests, the government has identified key thresholds 
— in effect pass or fail judgements — that can be used to measure the academic 
success of all pupils. For pupils leaving primary school, the 2017 attainment 
thresholds are: 


attaining the ‘expected’ standard in reading, writing and mathematics 
attaining ‘greater depth’ in reading, writing and mathematics. 

For 16 year olds at secondary school, the 2017 attainment thresholds are: 
attaining a good pass in English and mathematics 


attaining a good pass in five subjects that are collectively known as the English 
Baccalaureate. 


Published ‘league tables’ of school level academic results have existed in England 
since 1992. In the years that followed, there was some dissatisfaction with a system 
that compared different schools using ‘raw’ attainment data and largely ignored 
the question of ‘value added’ compared to baseline on entry. In 1997, the report 

of the Value Added National Project was published, following almost two years of 
study from researchers at the University of Durham.’ A significant recommendation 
of this report was the need to establish Unique Pupil Numbers (UPN) that would 
allow the progress of individual pupils and groups of pupils to be tracked. Today's 
UPN system was introduced in 1999. UPN was eventually linked to an authoritative 
annual survey of the government school population known as the Pupil Level 
Annual Schools Census (PLASC)® which was introduced in 2002. 


Careful consideration had to made of data protection issues and processes put in 
place to secure the data and decide who should be granted access. 


Today, each student enrolled in a government school is given a UPN in the form 
of a unique 13-character identifier which is used to track attainment during the 
pupil's school career. The following guidance issued by the government in 2013, 
and updated in 2017, explained the purpose of the UPN system: 


‘The system was introduced to enable accurate and timely data sharing 
between schools, Local Authorities and central government, strengthening 
procedures for target setting and monitoring, policy evaluation and 
monitoring, thereby contributing to the raising of standards.” 


The design and use of the UPN system is governed by strict confidentiality 
rules that comply with data protection legislation. The system is compulsory for 
government schools and optional for private schools. 


Each student 
enrolledina 
government 
school is given 

a UPN inthe 
form of aunique 
13-character 
identifier which 
is used to track 
attainment during 
the pupil’s school 
career 


5Fitz-Gibbon, 1997 °In England PLASC was replaced by the National Pupil Database in 2006/7 - see https://nationalpupildatabase.wikispaces.com/ [accessed 11th Dec 2017] ’DfE, 2013 
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The National Pupil Database (NPD) is based on a national aggregate of UPN 

data. It tracks the academic performance of all pupils and can be analysed at 
national, regional, district and school levels. The patterns of performance can 
also be segmented at each of these levels according to the background of groups 
of pupils. This is because each UPN is ‘tagged’ so that in addition to academic 
attainment, the following information for pupils is recorded: 


e age 

¢ ethnicity 

° gender 

° socio-economic status as indicated by entitlement to free school meals 


e the wealth or poverty levels of the pupil's immediate neighbourhood as indicated 
by each pupil's ‘post code’ which indicates precisely where they live 


¢ first language 
e whether the pupil has any recognised Special Educational Needs. 


The data infrastructure is supported by technology: a public system for parents 
and the community, and a confidential system for school staff and inspectors. 


Today, any parent or other member of the public can access the overall school 
performance tables® and look at a range of key information about any school in 
England. This insistence upon public transparency is intended to: 


¢ apply pressure on professionals to improve performance 


e enable parents to access detailed school performance information when choosing 
a school for their child to attend. 


Below is an example of the information about schools made available to parents 
enabling them to compare performance. The website also links to inspection 


FIGURE 1: AN EXAMPLE OF INFORMATION MADE AVAILABLE ABOUT SCHOOLS 


ese compare-school-performance. service gov.uk 


a> GOV.UK Find and compare schools in England 


10-8 This is anew service = your feedback will help us to improve it. 


Add a school or college to start your comparison list Add this schoo! 


Q Search by fT a t 

Oxford Spires Academy 

Address Glanwitle Road, Oxford. OX4 2AU 

Add to 

Local authority: Oxfords conaeoniat 

Headteacher/Principal: Mrs Marianne Blake 

Age range Nto19 Map Soweme 20505 College 

Phase of education: Secondary and 16 to 18 preteen 

School type. Academy Sponsor Led Mainstream Oxford Goll Club @ 

Academy sponsor: CIBT Education Trust ? 

Gender of entry Mixed 

© Oxtord Spires Academy 

Denomination Does not apply 

Admissions policy Comprehensive . 
Unique reference: 136261 

Ofsted ratin: Yisted report sh 

. Bet | cists egnet © Google ee ee een ee 

Useful links: School website and Apply for a secondary 

Download data: Download full set of school data 


(CSV, 30B). (XLS, SOKB 


5See https://www.compare-school-performance.service.gov.uk/ [accessed 8th Dec 2017] 
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Any parent or 
other member 
of the public 
can access the 
overall school 
performance 
tables and look 
at arange of key 
information about 
any school in 
England 
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reports and gives access to detailed school performance data. For full details visit 
the compare school performance webpage.’ 


While school level results are published, detailed group and individual pupil 
performance data is confidential and available to school staff and school 
inspectors only. Teachers and inspectors use a single secure access website. This 
was known as RAISEonline until it was replaced by the Analyse School Performance 
website from 1 August 2017."° Secure access ensures protection of sensitive pupil 
data through the use of username and password access. 


The full name for RAISEonline was Reporting and Analyses for Improving School 
Self Evaluation. It was a web-based system providing data to support school 
self-evaluation and to assist data informed school inspection. The intended users 
were school staff, Ofsted inspectors and other professional stakeholders, but not 
the public. Access to the system was protected by usernames and passwords. This 
access was differentiated according to the user so that only school staff could see 
pupil identities, whilst other users of the system had access to the vast majority of 
the data but were denied access to the pupil identities. 


The revised web-based data system, Analyse School Performance, can only be 
accessed from the government's secure access site. 


°The webpage can be accessed at https://www.compare-school-performance.service.gov.uk/school/136261 [accessed 8th Dec 2017] 
10See http://www.forschoolseducation.co.uk/analyse-school-performance-raiseonline-replacement/ [accessed 11th Dec 2017] 


While school 

level results 

are published, 
detailed group and 
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performance data 
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available to school 
staff and school 


inspectors only 
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Lesson 2: English experience 
demonstrates the risks of an over- 
simplistic approach to school 
performance data. 


Experience in England has highlighted some of the potential but also some of 
the possible pitfalls of national data systems. 


Performance 
The notion of a single indicator, or a very small group of indicators, able to indicators make 
provide the basis for an overall judgement of school quality may have some possible further 
attractions to policymakers, but schools are complex organisations and questions 
good schools must ensure multiple outcomes. School performance needs to about aschool’s 
be measured using multiple dimensions and this requires different types of performance and 
indicators. effectiveness, but 


by themselves are 
not sufficient to 


It is also important to see performance indicators as the beginning rather than 
the end of a reflection on school effectiveness. Performance indicators make 
possible further questions about a school's performance and effectiveness, but 
by themselves are not sufficient to draw definitive judgements ‘from a distance’. 


draw definitive 
judgements ‘from 


a distance’ 
Binary pass or fail measures are often used in data systems, and we call these 


measures ‘threshold indicators’. Such threshold indicators are not without value. 
They have proved useful in clearly articulating government expectations for 
student outcomes. However, while threshold measures are easily understood, 
the resulting data can be misleading. Schools can have different data in terms 
of threshold performance but drilling down into the underlying pupil data may 
show that the difference is explained by the performance of a very small number 
of pupils, and that the actual differences between the schools are much smaller 
than it first appears. This is particularly true for small schools. Threshold data 
does not always provide a very nuanced view of performance. Knowing that, for 
example, 70% of pupils have attained the ‘expected’ standard tells us nothing 
about how much better than ‘expected’ the pupils attained, and it tells us little 
about the specifics of the attainment of the 30% of pupils who did not meet the 
‘expected’ standard. Did they fail narrowly to meet the target? Or was there a 
wider variety of performance levels within this group? 


The use of averages is an alternative to a threshold measure but averages can 
also be misleading. Average attainment indicators, such as ‘average score or 
grade attained’, offer the advantage of describing all the outcomes for a pupil or 
school regardless of whether any outcome met a particular standard. However, 
this makes the measure less accessible for a range of users. 


Consider the following example (Table 1) in which five pupils score outcomes on 
a scale of 1 to 8 and a good pass is a score of 5 or more: 
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TABLE 1: THE NECESSITY TO REQUIRE BOTH THRESHOLD AND AVERAGE MEASURES, 
ILLUSTRATED BY FIVE PUPILS SCORE OUTCOMES ON A SCALE OF 1TO 8 


School 1 School 2 
Measure: Pupil outcomes 2,3, 4,5, 6 1,1,5,5,8 
Threshold measure: 40% 60% 


% attaining a good pass 


Average score (2+3+4+5+6)/5 = 4 (1414+5+5+8)/5 = 4 


Although the above example may seem trivial, it illustrates the necessity 
of requiring both measures — threshold and average - in order to better 
understand performance. 


Headline whole school data often hides important internal variations. A mature 
approach to evaluating performance across a system like a school demands a 
range of indicators which enable us to answer questions such as ‘what was the 
average attainment across particular group of pupils?’ 


Both threshold measures and averages fail to show the level of progress that 
students have made during a course of study. Headline aggregate threshold 
measures and averages also fail to shed light on the relative performance 

of different groups of students such as boys or girls, or those from different 
socio-economic backgrounds. 


During the last three decades, there has been considerable discussion in 
England about the relative merits of different types of measure and this has 
led to some interesting but also controversial experimentation in the field of 
‘value added’ data which is described in lesson 3, overleaf. 
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Lesson 3: In England, there has been an 
important and lively debate about ‘value- 
added’ measures which take into account 


the starting points of pupils. 


The academic performance of a school is determined in part by the pre-existing 
achievements of pupils before they join the school. Teachers have, of course, 
always been aware of this and that some schools serving socially advantaged 
communities, and in which the academic starting points of pupils was relatively 
high, can have lower rates of progress than other schools with poorer initial 
intakes. Schools with higher rates of progress compared to baseline attainment 
are surely more effective. There is a strong case, therefore, for an adjustment of 
attainment scores based upon a baseline of the intake achievements of pupils. This 
adjustment, leading to ‘value-added’ scores for schools, is now a well-established 
principle and accepted formally by the government in England, but the exact 
mechanisms for calculating ‘value added’ remain contested. 


The first ‘league tables’ of school performance in England published in 1992 did 
not include a ‘value added’ dimension. There was a public debate about the need 
for a fairer ‘value added’ approach throughout the mid-1990s. The University of 
Durham published a report based on a Value Added National Project." This report 
called for ‘the development of a national system of value-added reporting for 
schools based on prior attainment, which will be statistically valid and readily 
understood’ After a pilot phase for secondary schools from 1998 onwards, ‘value 
added’ scores were published for all secondary schools in England in 2001 and for 
all primary schools from 2003.” 


A National Audit Office report in 2003 suggested that this was not enough and 
recommended that school performance information should take into account not 
just prior attainment, but also ‘other external influences on performance’, based on 
UPN data, which provided data about the characteristics of individual pupils such 
as ethnicity, poverty levels or gender. In 2005, following a process of consultation 
involving academics, statisticians at the DfE, schools, local government and 
others, a ‘Contextual Value Added’ (CVA) score was published on a pilot basis in 
the performance tables. CVA results for all secondary schools were published in 
2006. The CVA calculation adjusted the ‘estimate’ for the performance of pupils by 
considering a range of factors including a pupil's gender, date of birth, ethnicity, 
special educational needs status and poverty levels. Many considered this to bea 
fairer method for evaluating school effectiveness, and proved useful in challenging 
schools with advantaged intakes whilst providing a measure that offered schools 
with very challenging intakes a way of demonstrating their effectiveness with their 
pupils. With CVA scores, schools could no longer argue that a set of results was 
poor because there were more boys than girls that year or that they had a high 
proportion of pupils with special educational needs. 


"Fitz-Gibbon, 1997 “DfEE, 1998; Ray, 2006 “The National Audit Office, 2003 
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From the beginning, the CVA system was controversial. Some people thought that 

the CVA formula was too complex and mysterious, understood only by statisticians. 

The earlier Durham University National Project had advocated simple statistical 

approaches and not more complex, multi-level models such as CVA. Others 

thought that the CVA approach encouraged lower expectations for pupils from 

different backgrounds. They argued: why should we have lower expectations based As the CVA 


on gender or ethnicity? approach was 
implementeda 


growing concern 
developed that 
some schools 


As the CVA approach was implemented a growing concern developed that some 
schools were playing or ‘gaming’ the system. Some ‘contextual’ information could 
be altered. An example was special educational needs categories for pupils, which 
at the time was subject, to some extent, to alteration at the discretion of school 
staff. There was a growing perception that some schools where choosing to over 
identify pupils on the special-needs action register, which was known to have the 
beneficial side effects of improving the CVA score for a school. Additionally, some 
schools began entering pupils for what were perceived by some to be ‘easier’ 
qualifications in the examinations for 16 year olds, which also had the effects of 
improving CVA scores. 


were playing 
or 'gaming' the 
system 


In 2010, the government policy paper ‘The Importance of Teaching’, called for an 
end to the CVA measure in England and it was phased out from 2011. In its place, 
the government developed new ‘value added’ measures for secondary schools. 
The Progress 8 measure that is used for secondary schools today has reverted to 

a methodology based on prior attainment without any reference to the particular 
background of individual students in the school cohort. 


Recent guidance® on the use of performance data recognises the significance of 
‘value added’ but takes a much a simpler approach than that adopted during the 
CVA years: 


‘In the performance data, we group pupils at key stage 2 and key stage 4 as 
low, middle or high attainers, depending on their attainment at the end of key 
stage 1 and key stage 2.’ 


‘You should consider how well a school’s low, middle or high attainers are 
progressing. This is an indication of whether a school is equally effective at 
helping all of its pupils meet their potential rather than, for example, just high 
attainers.’ 7 


“DFE, 2010 “DFE, 2016b "Ibid. ‘Ibid. 
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Lesson 4: There is a need fora 


national accountability system that 


evolves over time. 


In an ideal world, changing official key indicators for the measurement of school 
quality should be avoided. Teachers and headteachers can be annoyed by frequent 
change, especially if they are unconvinced by the rationale behind any change. 
New indicators may require a lead-in time of several years before outcomes can 
be measured as it takes time for pupils to move through the system. Substantial 
changes to indicators also make any national time series analysis difficult, thereby 
limiting the scope for the authoritative evaluation of the effect of national policy 
changes over a long period of time. 


Despite these caveats, there can be a strong case for a national education data 
system to be modified. Over the last twenty years, the system in England has 

been reformed on several occasions. New pupil assessment activities have been 
added; one relatively recent example is the Phonics Screening Check that was 
introduced for the first time in 2012. This is a compulsory assessment of all 
children in government schools in Year 1 - typically aged 6 — to ascertain their 
reading proficiency. Similarly, in 2014, the government reformed the assessment 
system and abolished the so-called ‘Levels’ (the numerical grading system that was 
used to assess pupils at ages 7 and 11) on the grounds that they promoted a rather 
simplistic understanding of student progress. 


In the remainder of this section, we reflect on some of the ways that the analysis of 
secondary school performance data has been changed in recent years, in response 
to both the priorities of politicians and a desire to provide a more comprehensive 
system that went beyond threshold measures. 


A case study in the evolution of the English data system 


There was a change of government in the UK in 2010. The new Cabinet minister for 
Education in England immediately established a new school performance measure 
called the English Baccalaureate (EBacc). It allowed people to see how many pupils 
in each school achieved a high grade across five traditionally ‘academic’ subjects 
which the minister considered to be ‘core’ subjects: 


English 

Mathematics 

History or geography 
Science 


a foreign language. 


New indicators 


may requirea 


lead-in time of 
several years 
before outcomes 
can be measured 
as it takes time 
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The reform was driven by a concern that too many students were stopping their 
study of a foreign language or a humanities subject, in the form of history or 
geography, at the age of 14. There was, of course, an element of subjectivity and 
political judgement in the decision to privilege these five subject areas. Advocates 
of other curriculum areas — such as the arts and physical education - were 

very disappointed. 


The story of the introduction of the EBacc is an interesting case study in how 
performance indicators can change behaviour. Some, but not all, schools altered 
their curriculum in response to the EBacc performance measure. In 2016, The 
Sutton Trust, a research charity, assessed the impact on pupils in those schools 
which had significantly adjusted their educational offer following the introduction 
of the EBacc.* Their view was positive. They studied 300 secondary schools - 
termed curriculum change schools - which had transformed their Key Stage 4 
curriculum between 2010 and 2013 in response to the new government policy, 
achieving a rise in the proportion of pupils entering the EBacc from 8% to 48%. 
In the view of The Sutton Trust, able students from disadvantaged backgrounds 
benefitted from this change because it provided them an opportunity to achieve 
in subjects highly valued by top universities. 


The Attainment 8 and Progress 8 reforms 


For many years, the highest profile secondary school performance indicator 
was a percentage measure of pupils attaining five ‘good’ General Certificates of 
Secondary Education (GCSEs) by the age of 16. The GCSE qualifications were of 
a ‘good’ standard if a pupil attained a grade A (later including a new grade of A*), 
B or C only (the full range was A through to G). This 5+ A*-C percentage 
incentivised schools, teachers and pupils to attain these higher grades in five 
separate subjects, but has also had unintended consequences and accidentally 
introduced perverse incentives. 


The measure emphasised the importance of five subjects being passed at a good 
standard, but did not specify the subjects themselves. A further assumption was 
that a grade C and above was the same standard across all subjects and that any 
five subjects would do. School leaders in the system responded accordingly, 
maximising the 5+ A*-C grades to impress the local community and the school 
inspectorate. In some cases, schools realised that they could achieve this by having 
pupils succeed in subjects that were not necessarily of sufficiently high value 

and would not provide the best basis for further study or personal development. 

In other words, the pupils’ best interests were not always considered first and 
foremost. Furthermore, it became clear that some schools were seemingly doing 
well but on closer inspection essential subject areas such and English and maths 
were not being prioritised for all pupils. It is widely accepted that high standards in 
literacy and numeracy improve the life chances of students worldwide. Many pupils 
attaining 5+ good grades were not doing well in these core subjects areas. The 
government took action. In 2004 the accountability measure was redefined so that 
the five good qualifications had to include both English and mathematics. 


18 Allen and Thompson, 2016 
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This change seemed to encourage schools to prioritise English and maths, but 
those still seeing a need to play the system focused attention on the borderline 
grade C pupils instead. There was a strong incentive to focus on interventions for, 
and direct resource towards, those pupils close to the C/D borderline. There was 
much less incentive to focus interventions or teaching resource elsewhere. For 
example, on the A/B or F/G grade boundaries. There was also the widespread view 
that these measures emphasised ‘teaching to the test’ 


Trying to avoid or address perverse incentives in setting 
performance measures 


All performance indicators have the potential to introduce such perverse incentives 
and these concerns can only be mitigated by either introducing other additional 
indicators alongside or by rethinking the principles behind the accountability 
measures. 


Following the 2010 election and a change of government in England, as 

mentioned earlier, the accountability measures for English secondary schools were 

reconsidered. Fundamental assumptions about the curriculum were re-examined. 

The number of qualifications being used to hold schools to account was deemed 

to be too small, with only five counting. This was fundamentally revised into a new 

performance measure, now known as Attainment 8,!° which would count eight 

separate subjects. 

Following the 
2010 election 
and a change of 
government in 
England... the 
accountability 
measures for 
English secondary 
schools were 
Planning a large scale national change, such as the introduction of Attainment reconsidered 
8, required long term planning. Schools leaders needed time to adapt their 

curriculum, deploy specialist teachers and adjust timetables. The new measure 

changed behaviour at school level. It has succeeded in schools changing their 

curriculum to maximise the number of pupils entered for these qualifications, 

with the first national results published in 2016. 


Additionally, the qualifications themselves were reviewed. Many were judged 

to be of low value and removed from the list of qualifications approved for 
accountability purposes.*? However, schools were able continue to offer these 
qualifications if they wished. Of the eight high value qualifications that were to be 
counted, these had to include English, mathematics, science, a foreign language, 
a humanities subject and three others. Compared to other subjects, English and 
mathematics were given a double weighting in the calculation of each school’s 
overall performance. 


Attainment 8 is not a so-called ‘threshold measure’ based on a key mark or 
threshold that constituted a ‘pass or fail’ mark. Progress 8 does not count the 
percentage of pupils attaining certain grades with certain qualifications. A lesson 
learnt from the previous concentration on grade C and the perverse incentives it 
had created was that the new accountability measure should value all the grades 
a pupil attained and not simply those of a type. The introduction of point scores 
required grades to be categorised numerically rather than alphabetically, as had 
been the case previously. A new number-based grading system was introduced 
for the GCSE examination in 2016. 


Further details can be found at https://www.gov.uk/government/publications/progress-8-school-performance-measure [accessed 8th Dec2017] *°Wolf, 2011 
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TABLE 2: GRADE CONVERSION POINTS IN THE NEW ATTAINMENT 8 SYSTEM* 


Previous grade for the GCSE examination 


New point score (for teaching from 2016) 


A* 9 
A 7/8 
B 6 
C 5 
D 4 
E 5 
F 2 
G 1 


*The new point scores for GCSE qualifications are in the process of further change in order to give greater weight to the 


highest gradings. 


Table 3 explains the full range of quality indicators that are currently used to 
measure secondary school quality in published performance tables. 


TABLE 3: 2017/18 PERFORMANCE TABLES HEADLINE INDICATORS FOR SECONDARY SCHOOLS IN ENGLAND 


Indicator 


Type of indicator 


Description 


Attainment 8 


Average score 


Calculated across 8 qualifying 
subjects 


Progress 8 Value added score Compares the performance of pupils 
with similar pupils nationally across 
8 qualifying subjects 
English Threshold indicator Percentage attaining a good grade in 
Baccalaureate English, mathematics, two sciences, 
(EBacc) a humanity and a language 
English Percentage of pupils entered for A percentage of those entered for 
Baccalaureate the qualifying qualifications for the the EBacc (includes those who did not 
(EBacc) EBacc attain good passes) 
English and Threshold indicator (displayed as a Percentage of pupils attaining a good 


mathematics 


percentage) 


grade (5+) in English and mathematics 
only 


Destinations 


Count and percentage 


Statement about the percentage 
of students staying in education 
or employment after key stage 4 


Pupil groups 


Allindicators are shown for specific 
pupil groups in a school 


Prominence is given to 
disadvantaged pupils 


The headline data set out in Table 3 is published via the performance tables, which 
can be located on the ‘find and compare schools in England’ web pages.” 


1See https://www.gov.uk/school-performance-tables [accessed 11th Dec 2017] 
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FIGURE 2: AN EXAMPLE OF THE DEPARTMENT OF EDUCATION’S ‘FIND AND COMPARE SCHOOLS’ WEBPAGE 
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England transport to school 
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You can view and download: 


* exam and test results 
+ Ofsted reports 
* financial information 


Northern Ireland. Scotland and Wales have their own websites for 
comparing schools. 


Startnow > 


FIGURE 3: AN EXAMPLE OF COMPARATIVE SCHOOL DATA PRODUCED (IDENTITY OF SCHOOLS HIDDEN) 
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key stage and maths of above employment 
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Remove 
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SCHOOL 2 (197 of 205 
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SCHOOL 3. School (196 of 202 
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Rem 
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SCHOOL 4 School (176 of 185 
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CHAPTER 2: THE LESSONS LEARNED 


29 


CHAPTER 2: THE LESSONS LEARNED 


Lesson 5: Accountability measures have 
played a significant role in the development 


of school self-evaluation. 


During the evolution of the data system in England in recent years, people 
have often asked the question: does the data only exist to make accountability 
possible, or can it also be used for the purpose of internal reflection and 
improvement planning? If the data system has these two purposes, then how 
can we ensure that the system provides useful management information for 
school leaders? 


There is no doubt that many school leaders in England have made very 
substantial use of the national data systems to support their work as agents of 
change at school level. The work of Melanie Ehren, at the Institute of Education, 
UCL, has shown how many school leaders in England have internalised 
expectations derived from the external accountability infrastructure. In Ehren’s 
study of the impact of accountability in six different European jurisdictions, 

she described the positive direct effect of accountability mechanisms, such 

as inspection, on ‘setting expectations’ and the marked extent to which 
headteachers in England appeared to view the external accountability landscape 
as a source of expectations that could be used internally for the purpose of 
self-evaluation and improvement.” Ehren also described how the accountability 
framework could be used to improve self-evaluation and build school level 
capacity. For Ehren, school leaders in England typically saw their role as one of 
‘institutionalising’ norms derived from external accountability. 


This analysis is also supported by recent Education Development Trust research 
into the improvement of schools in London, and the rapid turnaround of some 
schools in England. In our 2016 study of London schools, the internal use of data 
was highlighted as a key driver of school transformation: 


‘One of the most important developments in London since 2000 has been 
the growth in the use of education performance data and improved data 
literacy among education professionals. In extensive interviews with experts 
and serving teachers we found virtual unanimity in the identification of data 
analysis and data literacy as key to the transformation. This preoccupation 
with data was not the exclusive property of any particular group and all the 
major initiatives seemed to have strong foundations in the use of educational 
metrics. The different actors in the London story are therefore linked by 

a common preoccupation with the effective use of educational data as 

an instrument for transformation. The use of data made possible a better 
concept of school leadership, based on a relentless focus on the quality of 
learning outcomes and the action needed to improve these outcomes.’ * 


Ehren, et al., 2015 **McAleavy and Elwick, 2016, p.10/11 
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While the internal use of data has been a positive element within the story of the 
improvement of government schools in England, the current data systems give 
greater priority to the requirements of external scrutiny and pressure rather than 
internal self-improvement. Performance against the current Attainment 8 and 
Progress 8 measures for secondary schools is, to some extent, beyond the control 
of school leaders because it is a retrospective, comparative measure. People often 
talk about the importance of monitoring and evaluation as if the two concepts, 
‘monitoring’ and ‘evaluation’, were essentially synonyms. This is not the case. 
Schools need to evaluate retrospectively, but they need to monitor in real time. 
Accountability measures need to be defined in such a way that schools can monitor 
their performance in real time. This is necessary if professionals are to have the time 
to intervene to adjust performance before students undertake summative tests. 


Data and data literacy are preconditions to improvement but they are not sufficient. 
Whilst performance data can alert school leaders to problem areas, schools also 
need diagnostic skills to create answers to the questions: ‘Why did this happen?’ 
‘What should we do to solve this problem?’ The data alone will not provide answers 
to these questions. So, schools need data literacy but they also need access to 
evidence-based solution design. 


School level capacity to make use of externally provided data depends upon 
internal data literacy, and effective training for effective data literacy. 


Data literacy requires the ability to go beyond the obvious explanation. Consider 
the case below, which is the better school? 


TABLE 4: COMPARISON SHOWS THE NEED FOR DATA LITERACY TO GO BEYOND OBVIOUS EXPLANATION 


Measure School A School B 

% of pupils attaining ‘expected’ standard 70% 85% 
Average attainment of all pupils regardless of 102.1 101.9 
whether they achieve a threshold such as ‘expected’ (Average (Average 
(measured as a scaled score) attainment) scaled score) 


Data literacy at school level in England is supported both by government action 
and a market for private support, provided on a commercial basis. Private providers 
of support for data analysis help school systems to undertake target setting and 
improvement planning. One significant example of a private provider of support is 
the data support provided by The Fischer Family Trust (FFT). The FFT Aspire web- 
based system is now widely used, particularly in secondary schools, for school, 
subject and pupil target setting data. Issues of ethnicity, disadvantage, and special 
educational needs, among other factors, were intentionally excluded from the 
model because of the possible lowering of expectations that consideration of 
these factors might encourage. 


Figure 4, opposite, provides an example of benchmarks generated by FFT Aspire** 
for some of the subjects taken by a single pupil. The prior attainment, date of birth 
and gender used to generate these benchmark estimates is shown at the top. The 
typical outcome for the grade in each subject is shown with chance of a higher 
grade and the risk of a lower grade. 


4 See https://fft.org.uk/ - accessed 11th Dec 2017 
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FIGURE 4: FFT BENCHMARKS 
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In England, several relatively small companies have offered products to the school 
market to support the collection and processing of school pupil performance 
data. These companies provide School Management Information Systems (SMIS) 
designed to assist schools in the capture, storage, processing, analysis and 
reporting of data. Some SMIS provide specialist support in particular areas, such as 
the analysis of ‘attendance’ or ‘item level’ question response. Other SMIS products 
provide comprehensive support or reinterpret school and national data. They aim 
to provide the full range of monitoring and tracking reports that schools perceive 
they need. 


33 


CHAPTER 2: THE LESSONS LEARNED 


Lesson 6: Schools can benefit from 
comparing their own students’ 
performance with that of the national 
database through item level analysis. 


One significant area for any national data system is the provision of advice on 
pupil performance at the level of individual test questions - known as ‘items’. 
Item level analysis can help teachers to understand specific and characteristic 
barriers to learning. This has recently been an area of reform in England. For the 
first time, in 2016, all the national Key Stage 2 tests in reading, mathematics and 
English grammar, punctuation and spelling, had the item level data collected 
electronically for each test question. 


Item level 
analysis can 
help teachers 
to understand 
specific and 
characteristic 
This item level data was published, with accompanying analysis to help teachers. barriers to 

The analysis allowed schools to compare their own pupils’ performance with the learning 
national performance in the individual test questions. The same information — at 
the level of individual students - was also made available to those secondary 
schools receiving primary pupils as their new intake. For the first time, secondary 
school teachers were able to access this item level detailed information about 
what pupils could and could not do. 


The following example of ‘item level’ analysis is taken from the 2017 release of 
data in ‘Analyse School Performance’. 


FIGURE 5: AN EXAMPLE OF ITEM LEVEL ANALYSIS 


Question level 
analysis Reading “A 


List of reports Based on the group of 100 pupils found 


Marks available 2 School% 3 National% 2 Difference 2 


Give / explain the meaning of words in context 10 61 76 15 
Retrieve and record information / identify key 14 50 73 -23 
details from fiction and non - fiction 

Summarise main ideas from more than one 2 68 75 7 
paragraph 

Make inferences from the text / explain and justify 22 28 59 -30 
inferences with evidence from text 

Identify / explain how information / narrative 1 33 43 -10 
content is related and contributes to meaning as a 

whole 

Identify / explain how meaning is enhanced through 1 31 53 -22 


choice of words and phrases 


Total 50 43 66 24 


2) Viewdatabyquestion [2] Viewdatabypunil [3 View test paper 


See http://www.forschoolseducation.co.uk/analyse-school-performance-raiseonline-replacement/ [accessed 11th Dec 2017] 
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Item level analysis of individual performance in primary school tests, facilitates 
detailed self-evaluation and potentially improves transition arrangements from 
primary to secondary schools. This is a new development and such data has only 
recently become available for national use in England. 
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Lesson 7: There should be an alignment 
between data-based support for internal 
schoolimprovement and external school 
improvement through inspection. 


England's school inspectorate, the Office for Standards in Education (Ofsted), 

is a key part of the accountability system in England, and relies heavily on the 
nationally-available data. Ofsted publishes detailed information about historic 
school performance, highlighting both overall school performance and data which 
focuses on the performance of distinct pupil groups. The inspection system is 
‘high-stakes’, and a wide range of judgements about the school are consequently 
published for schools, parents and the public. 


The inspection 
system is ‘high- 
stakes’,anda 
wide range of 
judgements about 


the school are 
Schools are currently judged in terms of four areas: pupil attainment, leadership consequently 
and management, behaviour, and teaching. There are strong links between published for 

the national pupil database (NPD) and the data that underpins the work of the schools, parents 
inspectorate. Data from the NPD, together with that collected by the schools, is and the public 
considered very carefully in the inspection process, and forms a key part of the 
judgements that are subsequently made about the school. Such is the pressure of 
and consequences of the inspection system, that schools engage in extensive data 
collection and analyses themselves to prepare and defend themselves during the 
challenging inspection process. 


Ofsted emphasise pupil progress and apply this analysis to a range of pupil groups, 
including: disadvantaged pupils, those with special educational needs and gender. 
These groups are currently considered to be national priority groups for closing 
the attainment gaps that are known to persist between the disadvantaged and 
advantaged. 


Figure 6 illustrates the first page of an Ofsted inspection dashboard for inspectors, 
analysing quality in a secondary school. 


Ofsted share their ‘inspection dashboard’ with both their own inspectors and 
schools. Both the data and the analysis are shared with schools so that both 
inspectors and school staff have a common understanding of the data. Figure 7, 
overleaf, illustrates this data. 
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FIGURE 6: EXAMPLE OF INSPECTION DASHBOARD CONTENT FOR A SAMPLE SCHOOL” 


Inspection 
dashboard 
(overview) 


Strengths 
in 2016 


Weaknesses 
in 2016 


2016 

Progress 8 
Above? 

Coasting elements 


Above? 


The inspection dashboard is designed to show at a glance how well 
previous cohorts demonstrated characteristics of good or better 
performance. It contains a brief overview of progress and attainment 
for 2016 and other data for the last three years. It shows progress 
first, including from the main starting points. 


It includes the key groups: disadvantaged pupils, those who have 
special educational needs (SEN), girls and boys. Achievement of 
disadvantaged pupils is compared with the national performance of 
other (non-disadvantaged) pupils, overall and by prior attainment. 
SEN group progress is compared with the national for all pupils, 
which is zero. 


The front page summarises strengths and weaknesses based on 
only the 2016 data shown in the dashboard. The strengths give an 
indication of some features of good or better performance in 2016, 
highlighting consistency across starting points and subjects. 


¢ Progress 8 was not significantly below average* overall or for any 
prior attainment group. 


¢ Progress 8 was not significantly below average* overall or for any 
prior attainment group in English or mathematics. 


*and not well below average. 


¢ Progress 8 in English or mathematics was significantly below average 
and in the lowest 10% for the group: disadvantaged high. 


¢ Progress was significantly below average and in the lowest 10% in 
at least one of EBacc or open elements or science, languages or 
humanities for the group: disadvantaged middle. 


¢ Attendance was low for the group: SEN support (in the lowest 10%). 


¢ Persistent absence was high for the group: SEN support (in the 
highest 10%). 


Weaknesses are indicated for cohorts of at least six. Data for very 
small groups should be treated with caution. Where a group is 
identified as in the highest or lowest 10%, it has been compared with 
the highest or lowest 10% of schools based on the figures for all 
pupils, and not the figures for the group nationally. Where progress 
is said to be ‘not well below average, it is not in the lowest 10%. In 
progress strengths, ‘significantly’ refers to statistical significance 
based on a 95% confidence interval. 


School National floor National coasting 
-0.09 -0.5 -0.25 

N/A Yes Yes 

2014 2015 2016 

Yes Yes Yes 


°6 See https://www.gov.uk/government/uploads/system/uploads/attachment_data/file/633596/16_to_19_ 
study_programmes_inspection_dashboard_anonymised_school_final_June_2017.pdf for details 
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FIGURE 7: AN EXAMPLE OF DATA DASHBOARD CONTENT 


In each group, data is shown overall and for pupils with low, middle and high prior attainment. National figures are shown by dots. 
The tables show the percentage of the cohort that attained the grade C threshold. The difference from national is shown as the 
number of pupils it represents and, for prior attainment groups, shaded green if three or more above red if three or more below. 


ALL 
% ATTAINED ENGLISH AND MATHEMATICS, - 4+ PUPILS ABOVE NATIONAL AVERAGE DISADVANTAGED 
ALL/DISADVANTAGED/OTHER COMPARISON 4+ PUPILS BELOW NATIONAL AVERAGE OTHER 
% DIFFNO. COHORT ENTRYNO. ENTRY % © NATIONAL % 
a 7 174 170 98 62 
<4 
& 30 17 44 42 95 of 
Oar cs x) 130 128 98 70 
15 2 55 52 95 i 
3 5 1 19 18 95 14 
19 2 36 34 94 14 
ae 9 92 91 99 63 
a 43 5 23 22 96 68 
ames 10 69 69 100 68 
100 1 25 25 100 96 
3 100 0 2 2 100 97 
100 0 23 23 100 97 
0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% 
% ATTAINED ENGLISH AND MATHEMATICS, 4 tts 
GENDER COMPARISON 1) 44 PUPILS ABOVE NATIONAL AVERAGE BOYS 
% DIFFNO. COHORT ENTRYNO. ENTRY% © NATIONAL % 
z 58 7 85 82 96 66 
= 58 0 89 88 99 58 
= 4 0 27 25 93 12 
ea 1 28 27 96 10 
a 69 1 42 41 98 66 
[=] 
= 7% 8 50 50 100 59 
= 100 0 16 16 100 97 
= 100 0 9 9 100 95 
0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% 


SEN WITH EHC/STATEMENT 
SEN SUPPORT 


% ATTAINED ENGLISH AND MATHEMATICS, 


SPECIAL EDUCATIONAL NEEDS COMPARISON NO SEN 
% DIFFNO. COHORT ENTRYNO. ENTRY % ~ NATIONAL % 
= 20 - 10 7 70 - 
= 
& 21 - 28 28 100 - 
Oo 68 « 136 135 99 69 
0 - 7 4 57 - 
= 
i) 14 - 22 22 100 - 
19 - 26 26 100 15) 
y 67 - 3 3 100 - 
8 50 - 6 6 100 = 
= 
75 - 83 82 99 65 
m = 0 0 ps es 
= 
Ss - - 0 0 = = 
= 
100 - 25 25 100 96 
0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% 
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Final thoughts 


CHAPTER 3: FINAL THOUGHTS 


The purpose of this overview has been 
to describe England pupil level data 
story with the intention of sharing some 
of the most important lessons that have 
been learned. 


The development of a data rich system that allows analysis of performance at 
pupil level is a core pillar of the school improvement infrastructure in England. 


; : We hope this 
This report has shown the development of the system has not been without : 3 
report will provide 
challenges and dangers. : : 
interesting 
While it is not our intention to suggest that England's system should be seen reading for 
as a perfect example and used simplistically as a blueprint for others we hope policymakers in 


this report will provide interesting reading for policymakers in other countries other countries 
where a national pupil level data system is being developed or refined. By sharing 
England's story we hope that others may be able to avoid some of the mistakes 
and unexpected consequences that were made in the last three decades of 
development in England. 


We invite readers to visit our website and explore our rich library of research and 
insights - www.educationdevelopmenttrust.com/research. 
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Education Development Trust... we've changed from CfBT 


We changed our name from CfBT Education Trust in January 2016. Our aim 
is to transform lives by improving education around the world and to help 
achieve this, we work in different ways in many locations. 


CfBT was established nearly 50 years ago; since then our work has naturally 
diversified and intensified and so today, the name CfBT (which used to stand 
for Centre for British Teachers) is not representative of who we are or what 
we do. We believe that our new company name, Education Development 
Trust — while it is a signature, not an autobiography — better represents both 
what we do and, as a not for profit organisation strongly guided by our core 
values, the outcomes we want for young people around the world. 
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